Skip to content

Auto-diagnostics: failures report themselves, with a hard cost ceiling - #1128

Closed
arul28 wants to merge 14 commits into
mainfrom
ade/clerk-log-triage
Closed

Auto-diagnostics: failures report themselves, with a hard cost ceiling#1128
arul28 wants to merge 14 commits into
mainfrom
ade/clerk-log-triage

Conversation

@arul28

@arul28 arul28 commented Aug 19, 2026

Copy link
Copy Markdown
Owner

When ADE hits a serious failure, the already-redacted diagnostic report now uploads automatically — no more asking users to click Report issue or run terminal commands. Follow-up to #1122/#1127; closes the loop the Fozaan incident opened: next time, the report is in the bucket before the user texts us.

Triggers (each one call at the existing failure point)

Terminal recovery diagnosis · renderer crash · update-step failure · pairing auto-recovery giving up · publish failing >5 min. Reports carry auto: true + the failure code, so triage starts pre-sorted.

Trust guardrails

  • Same aggressive redaction as manual — only the when changed, never the what.
  • Settings → General → Privacy: "Share diagnostics with ADE when something breaks" (default on, machine-scoped, hidden on web; brain reads the same file).
  • A toast on every send — "A diagnostic report was sent to ADE · View · Turn off" — with an ack contract: pending means "no renderer acknowledged rendering this," so headless/brain sends surface at the next window, and an acknowledged toast never repeats.
  • Client budgets: 1 per failure class per 24h, 3/day per install (one shared desktop+brain ledger, mkdir-lock, fail-closed).

Hard cost ceiling (the worker is the sole bucket writer)

Fleet-wide 400/day budget in D1 (migration 0009): single-statement atomic claim, refund on failed store, fail-closed 503, DIAGNOSTICS_DAILY_GLOBAL_LIMIT=0 kill switch, distinct 429s (per-user vs fleet). With the 512KB size cap and 30-day bucket lifecycle: worst-case ~6GB steady state — inside R2's free tier at any user count. Per-identity quota only counts stored reports.

Also

ade doctor gains a "Diagnostics sharing" row; docs in lockstep; iOS/TUI verified not-applicable.

/quality: 2 passes, 18 findings, all applied, empty gate. /test: parity complete, all affected suites green (desktop 138, CLI 133, worker 161).

🤖 Generated with Claude Code

ADE   Open in ADE  ·  ade/clerk-log-triage branch  ·  PR #1128

arul28 and others added 14 commits August 18, 2026 21:04
…p phantom machines, log refusals

Two-phase grant redemption (reserve/consume/release with a 60s crash bound,
never extending expiry), device-supersede dedup returning supersededMachineKeys
under the fresh-interactive-auth bar, and structured refusal log lines
(key prefixes only) for support. Migration 0006 is additive and safe in either
deploy order.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sals

Durable identity writes (fsync + .bak), corrupt-file recovery preserves the
machineKey and re-mints only the secret, rotation budget persisted (2/24h) with
a 'needs reconnect' state when capped, and a pairing auto-recovery loop
(1m/5m/hourly, 3 per 6h persisted budget) that runs the repairMachinePairing
flow while a signed-in session exists. Consumes supersededMachineKeys.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nnected

Both credential store classes now expose their read state; githubService (and
the remote-runtime service) emit credentialStoreUnreadable instead of a
fresh-install-identical status, and every surface that said 'not connected'
now says the sign-in can't be read and routes to Repair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r preflight

classifySqliteOpenError buckets raw fs errnos (EDEADLK et al) as
storage_read_failed; bootstrap boot failures carry their classification instead
of a bare rethrow; the JSON-RPC -32603 fallback redacts system-level errors to
a referenced internal-error message logged for ade report-issue; dataless
cloud-placeholder preflight on the project DB; connection-pool retry now
classifies transport drops by error code instead of substring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ully heals

Per-account-salted hardware anchor (IOPlatformUUID / MachineGuid /
/etc/machine-id, sha256 with the account id — raw UUID never leaves the
process) sent on register; directory stores it (coalesce so a blank heartbeat
can't erase it) and supersedes stale rows matching device_id OR hardware_id
under the same fresh-auth bar. Migration 0007 additive and order-safe.

telemetry: make the invisible failure classes visible

ade_brain_action_failed (action_domain + shape-bounded error_code, deduped),
machine_removed, and machine_register_refused (edge-triggered refusal_code from
publisher health) — all strict-allowlist, codes only, never message strings.

diagnostics: one-click 'Send to ADE' for the redacted report bundle

POST /diagnostics/upload on the directory worker (512KB cap, 5/day limit,
R2-backed, never echoes), Send to ADE on the report-issue surface, and
ade report-issue --send that works with the brain dead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Correctness: anchor v2 folds the ADE home path so Stable/Beta/second-OS-user
installs stop superseding each other; auto-recovery honors a fresh (<10m)
deliberate removal; the false-premise win32 pre-unlink is gone (identity writes
go through writeFileAtomic with mode + scoped copy fallback); ERR_*/
MODULE_NOT_FOUND messages are redacted with a ref id; supersede folds the
custom machine name and runs as one D1 batch; diagnostics route hardening
(strict Bearer parse, 503 on unconfigured auth, remote-origin cross-site
rejection, no XFF quota identity).

Structure: one shared diagnostics upload client (renderer→main import removed),
directory.ts split along grant/relay/logging seams (1620→1085, handleRegister
~60 lines), action-domain allowlist imported not mirrored, shared refusal
decoder (403-only), shared credential read-state helper, coded-error wire
format has one home, store status projection extracted, dead getters deleted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…owing, anchor canonical paths, worker consolidation

The one regression: unlisted recovery codes with a brain-authored message now
surface that message as the headline (storage_read_failed keeps its move-out-
of-iCloud sentence). Latent: jsonrpc only strips a message prefix equal to the
error's own code; hardwareAnchor routes win32 homes through canonicalWindowsPath.
Structural: trustedOrigin/logging/jwks/fakeD1 consolidation in the worker,
refusal-code set built from the shared const, dead export removed, status
projection typed honestly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ion, anchor home fold memoized, jargon guard on promoted brain copy

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cs lockstep

TUI /report-issue gains send (same shared upload client, build-then-send so a
failed upload costs nothing); ade doctor stops discarding the directory-refusal
sentence and stops letting a stale relay probe mask the needs-reconnect state;
internal docs updated across sync-and-multi-device, storage-and-recovery,
pull-requests, onboarding-and-settings, ade-code, ARCHITECTURE, and logging.
iOS verified not-applicable per delta.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts:
#	apps/account-directory/src/directory.ts
#	apps/account-directory/test/directory.test.ts
#	apps/ade-cli/src/services/account/accountMachinePublisherService.ts
#	apps/desktop/src/renderer/components/settings/GitHubSection.tsx
#	apps/desktop/src/renderer/lib/githubIntegrationStatus.ts
#	docs/features/onboarding-and-settings/README.md
#	docs/features/pull-requests/README.md
#	docs/features/sync-and-multi-device/README.md
…d by unreadable store, TUI send arg parse, outage action label, migration comment

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…esigned

Storage-write failure surfaced as 502 with a structured log; diagnostics error
formatter is a redaction boundary; unreadable-store cleared on token writes and
snapshotted before awaits (both services); PublishToGitHub dialog stops offering
token replacement over an unreadable store and routes to Connections; durable
copy fallback staged atomically with mode; legacy credential read reasons
threaded; corrupt budget files read as spent; .bak-recovered identities defer
their rewrite; provider-accurate cloud-placeholder copy; send-button race
guarded; upload size measured on the serialized body.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Assemble the secret-shaped test literals from segments (the
diagnosticReport.test.ts convention) and fingerprint the introducing commit in
.gitleaksignore.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 19, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
ade Ignored Ignored Aug 19, 2026 4:52pm

@coderabbitai

coderabbitai Bot commented Aug 19, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 103 files, which is 3 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f919f8cb-4ce7-4d89-89a6-d6ce19aefbf1

📥 Commits

Reviewing files that changed from the base of the PR and between 064307b and c52bd6f.

⛔ Files ignored due to path filters (8)
  • docs/ARCHITECTURE.md is excluded by !docs/**
  • docs/features/ade-code/README.md is excluded by !docs/**
  • docs/features/onboarding-and-settings/README.md is excluded by !docs/**
  • docs/features/pull-requests/README.md is excluded by !docs/**
  • docs/features/storage-and-recovery/README.md is excluded by !docs/**
  • docs/features/sync-and-multi-device/README.md is excluded by !docs/**
  • docs/features/sync-and-multi-device/push-notifications.md is excluded by !docs/**
  • docs/logging.md is excluded by !docs/**
📒 Files selected for processing (103)
  • .gitleaksignore
  • apps/account-directory/README.md
  • apps/account-directory/migrations/0007_pairing_grant_reservations.sql
  • apps/account-directory/migrations/0008_machine_hardware_anchor.sql
  • apps/account-directory/src/activityRelay.ts
  • apps/account-directory/src/callerToken.ts
  • apps/account-directory/src/deviceAuthorization.ts
  • apps/account-directory/src/diagnostics.ts
  • apps/account-directory/src/directory.ts
  • apps/account-directory/src/index.ts
  • apps/account-directory/src/logging.ts
  • apps/account-directory/src/pairingGrants.ts
  • apps/account-directory/src/trustedOrigin.ts
  • apps/account-directory/test/deviceAuthorization.test.ts
  • apps/account-directory/test/diagnostics.test.ts
  • apps/account-directory/test/directory.test.ts
  • apps/account-directory/test/fakeD1.ts
  • apps/account-directory/test/helpers.ts
  • apps/account-directory/test/jwks.ts
  • apps/account-directory/test/machineSupersede.test.ts
  • apps/account-directory/test/pairingGrants.test.ts
  • apps/account-directory/test/trustedOrigin.test.ts
  • apps/account-directory/wrangler.jsonc
  • apps/ade-cli/README.md
  • apps/ade-cli/src/bootstrap.ts
  • apps/ade-cli/src/cli.test.ts
  • apps/ade-cli/src/cli.ts
  • apps/ade-cli/src/commands/doctor.test.ts
  • apps/ade-cli/src/commands/doctor.ts
  • apps/ade-cli/src/commands/reportIssue.test.ts
  • apps/ade-cli/src/commands/reportIssue.ts
  • apps/ade-cli/src/headlessLinearServices.test.ts
  • apps/ade-cli/src/headlessLinearServices.ts
  • apps/ade-cli/src/jsonrpc.test.ts
  • apps/ade-cli/src/jsonrpc.ts
  • apps/ade-cli/src/services/account/accountMachinePublisherService.test.ts
  • apps/ade-cli/src/services/account/accountMachinePublisherService.ts
  • apps/ade-cli/src/services/account/hardwareAnchor.test.ts
  • apps/ade-cli/src/services/account/hardwareAnchor.ts
  • apps/ade-cli/src/services/account/machinePairingAutoRecovery.test.ts
  • apps/ade-cli/src/services/account/machinePairingAutoRecovery.ts
  • apps/ade-cli/src/services/credentials/credentialStore.test.ts
  • apps/ade-cli/src/services/credentials/credentialStore.ts
  • apps/ade-cli/src/services/projects/machineLayout.ts
  • apps/ade-cli/src/services/sync/brainMachineSyncStores.ts
  • apps/ade-cli/src/services/sync/syncCloudRelayStatus.ts
  • apps/ade-cli/src/services/sync/syncCloudRelayStore.test.ts
  • apps/ade-cli/src/services/sync/syncCloudRelayStore.ts
  • apps/ade-cli/src/services/sync/syncRouteHealth.ts
  • apps/ade-cli/src/services/sync/syncService.ts
  • apps/ade-cli/src/services/sync/syncTunnelClientService.test.ts
  • apps/ade-cli/src/services/sync/syncTunnelClientService.ts
  • apps/ade-cli/src/tuiClient/__tests__/reportIssue.test.ts
  • apps/ade-cli/src/tuiClient/app.tsx
  • apps/ade-cli/src/tuiClient/commands.ts
  • apps/ade-cli/src/tuiClient/reportIssue.ts
  • apps/desktop/src/main/services/account/accountBridge.test.ts
  • apps/desktop/src/main/services/account/accountBridge.ts
  • apps/desktop/src/main/services/adeActions/domains.ts
  • apps/desktop/src/main/services/adeActions/registry.ts
  • apps/desktop/src/main/services/analytics/productAnalyticsPolicy.ts
  • apps/desktop/src/main/services/analytics/productAnalyticsService.test.ts
  • apps/desktop/src/main/services/analytics/reliabilityTelemetry.test.ts
  • apps/desktop/src/main/services/analytics/reliabilityTelemetry.ts
  • apps/desktop/src/main/services/github/credentialReadState.test.ts
  • apps/desktop/src/main/services/github/credentialReadState.ts
  • apps/desktop/src/main/services/github/githubService.test.ts
  • apps/desktop/src/main/services/github/githubService.ts
  • apps/desktop/src/main/services/ipc/ipcTimeouts.test.ts
  • apps/desktop/src/main/services/ipc/ipcTimeouts.ts
  • apps/desktop/src/main/services/ipc/registerIpc.ts
  • apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.test.ts
  • apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.ts
  • apps/desktop/src/main/services/prs/prService.ts
  • apps/desktop/src/main/services/runtime/projectRecoveryService.ts
  • apps/desktop/src/main/services/state/durableFile.test.ts
  • apps/desktop/src/main/services/state/durableFile.ts
  • apps/desktop/src/main/services/state/kvDb.rebuildRecovery.test.ts
  • apps/desktop/src/main/services/state/kvDb.ts
  • apps/desktop/src/main/services/storage/cloudPlaceholder.test.ts
  • apps/desktop/src/main/services/storage/cloudPlaceholder.ts
  • apps/desktop/src/renderer/components/app/IntegrationBannerHost.test.tsx
  • apps/desktop/src/renderer/components/app/IntegrationBannerHost.tsx
  • apps/desktop/src/renderer/components/app/ProjectRecoveryScreen.tsx
  • apps/desktop/src/renderer/components/app/ReportIssueButton.test.tsx
  • apps/desktop/src/renderer/components/app/ReportIssueButton.tsx
  • apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.test.tsx
  • apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.tsx
  • apps/desktop/src/renderer/components/settings/GitHubSection.tsx
  • apps/desktop/src/renderer/lib/githubIntegrationStatus.test.ts
  • apps/desktop/src/renderer/lib/githubIntegrationStatus.ts
  • apps/desktop/src/renderer/state/appStore.test.ts
  • apps/desktop/src/renderer/state/appStore.ts
  • apps/desktop/src/shared/accountMachineRefusal.test.ts
  • apps/desktop/src/shared/accountMachineRefusal.ts
  • apps/desktop/src/shared/codedError.test.ts
  • apps/desktop/src/shared/codedError.ts
  • apps/desktop/src/shared/diagnosticsUpload.test.ts
  • apps/desktop/src/shared/diagnosticsUpload.ts
  • apps/desktop/src/shared/types/git.ts
  • apps/desktop/src/shared/types/productAnalytics.ts
  • apps/desktop/src/shared/types/recovery.ts
  • apps/desktop/src/shared/types/sync.ts

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@arul28

arul28 commented Aug 19, 2026

Copy link
Copy Markdown
Owner Author

Created against the wrong head (lane-registered branch); superseding with the correct branch.

@arul28 arul28 closed this Aug 19, 2026
@arul28
arul28 deleted the ade/clerk-log-triage branch August 20, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant